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Field of Invention 

The present invention relates to imaging, and more particularly, to a four-color 
mosaic tiling pattern and color interpolation. 

Background 

5 A simplified cross sectional view of an imaging system is illustrated in Fig. 1 . 

Optical system 102 focuses electromagnetic radiation onto a focal plane, which may be 
taken as Color Filter Array (CFA) 104 and pixel sensor array 106. The CFA is usually 
deposited over pixel sensor array 106 by photo-lithographic techniques well known to the 
semiconductor industry. Pixel sensor array 106 is an array of pixel sensors, where in 

10 general, a pixel sensor is any sensor which absorbs radiation and provides a signal 

indicative of the absorbed radiation. Pixel sensor array 106 may be, for example, an array 
of charge coupled devices (CCD), or an integrated array of CMOS (Complementary Metal 
Oxide Semiconductor) pixel circuits. A pixel circuit may comprise a photo diode, where 
photons absorbed by the photo diode generate electron-hole pairs, along with additional 

15 circuits to provide an electrical signal, either a voltage or current signal, indicative of the 
number of photons absorbed by the photo diode. Photons incident upon various pixel 
circuits are pictorially indicated by 108. 

The spectral content of electromagnetic radiation focused onto a focal plane 
depends upon, among other things, the imaged subject, the illumination of the subject, the 

20 transmission characteristics of the propagation path between the imaged subject and 

optical system 102, the materials used in optical system 102, as well the geometric shape 
and size of optical system 102. For consumer imaging systems, the spectral range of 
interest is the visible region of the electromagnetic spectrum. 

The CFA is an array of filters, usually contiguous and deposited over pixel sensor 

25 array 106 so that each pixel sensor is substantially sensitive to only the electromagnetic 
radiation passed by one filter. (A filter in the CFA may actually be a composite filter 
manufactured from two or more filters, so that the transfer function of the resulting filter 
is the product of the transfer functions of its constituent filters.) Each filter in the CFA 
passes electromagnetic radiation within some spectral range. For example, a CFA may be 
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composed of red, green, and blue filters, so that the pixel sensors may provide signals 
indicative of the visible color spectrum. 

If there is not an infrared blocking element somewhere in the optical chain, then 
infrared radiation (IR), typically considered to be light with a wavelength longer than 780 
5 nm, may also be focused upon the focal plane. Imaging sensors or devices based on silicon 
technology typically require the use of infrared blocking elements to prevent IR from 
entering the imaging array. Silicon-based devices will typically be sensitive to light with 
wavelengths up to approximately 1200 nm. If the IR is permitted to enter the array, the 
device will respond to the IR, and generate an output image signal. Since the purpose of 

10 an imaging system (in this context) is to create a representation of the visible light present 
in a scene, the IR may introduce a false response and distort the image. In a monochrome 
(black and white) imaging system, the result may be an obviously distorted rendition. For 
example, foliage and human skin tones may appear unusually light. In a color imaging 
system, the introduction of IR may distort the coloration and produce an image with 

15 incorrect and de-saturated color. 

A common method for preventing these difficulties is to use ionically colored glass 
or a thin-film optical coating on glass to create an optical element which passes visible 
light (typically from 380 nm to 780 nm) and blocks the IR. This element can be placed in 
front of the taking lens, located within the lens system, or it can be incorporated into the 

20 imager package. The principle disadvantages to this approach are cost and added system 
complexity. 

An alternative approach is to allow IR to enter the focal plane, and then remove 
the IR signal electronically. In this approach, the CFA will have some IR pass filters 
(which also substantially block visible light), and the pixel sensor array would have some 
25 pixel sensors responsive to IR only. The effect of IR upon an image signal is substantially 
reduced by electronically subtracting signals generated by IR pixel sensors from signals 
generated by pixel sensors responsive to both IR and visible light. 

It is therefore desirable to provide for a CFA with a suitable arrangement of IR and 
visible color filters, and to provide for a method of processing the resulting pixel signals to 
30 obtain proper color interpolation without the need of an IR blocking filter. 
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Brief Description of the Drawings 

Fig. 1 illustrates a simplified cross sectional view of an imaging system. 
Fig. 2 is a unit array of a four color mosaic pattern for the RGB color space. 
Fig. 3 is a circuit diagram of a pixel sensor. 
5 Fig. 4 is a signal timing diagram for the pixel sensor circuit of Fig. 3. 

Fig. 5 is a flow diagram for color component signal interpolation. 
Fig. 6 is a unit array of a four color mosaic pattern. 

Fig. 7 is a block diagram of a computer system to perform color interpolation. 
Fig. 8 is a unit aray of a four color mosaice pattenr for the CMY color space. 
1 0 Detailed Description of Embodiments 

A four color Red-Green-Blue-InfraRed (R-G-B-IR) tiling pattern for a CFA is 
described in which the ratio of green, red, blue, and IR filters is approximately 4:1:1 :2. (It 
may be approximate due to "edge" effects in the deposition of the CFA.) Fig. 2 illustrates 
a unit array of a tiling pattern according to an embodiment of the present invention, where 
15 R, G, B, and IR, denote read, green, blue, and IR pass filters, respectively. The tiling 
pattern is provided by repeating the unit array of Fig. 2 in both the row and column 
directions. The number of repetitions may be non-integral. For convenience, we shall refer 
to IR as a color, so that the tiling pattern based upon the unit array of Fig. 2 is a four- 
color mosaic pattern. It is also to be understood that the R, G, and B pass filters may also 
20 each pass IR. 

It is to be appreciated that the designations of rows and columns is arbitrary, and 
that the pattern of filters in Fig. 2 may be rotated about a perpendicular to the figure. For 
notational simplicity, we shall denote a pixel sensor as red, green, blue, or IR, if it is 
responsive to substantially only electromagnetic radiation passed by a red, green, blue, or 

25 IR pass filter, respectively. 

Let X{n,m) denote the output signal of a pixel sensor responsive to the pass filter 
of row-column position (n,m) . The output signal may, for example, be a voltage or a 
current. An embodiment of a pixel sensor circuit in which the output signal is a voltage is 
shown in Fig. 3, where the pixel sensor circuit comprises pixel diode 305 and associated 

30 circuitry. Photons captured by diode 305 are measured in the following way. A reset 
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voltage signal goes HIGH to switch nMOSFET 310 ON so that diode 305 is reverse 
biased to a reverse bias voltage V*. This creates a wide depletion layer within diode 305. 
While reset voltage signal is HIGH, a shutter voltage signal also is set HIGH to switch 
nMOSFET 315 ON so that capacitor 320 is charged to the reverse bias voltage of diode 
5 305. When diode 305 is impacted by a photon with sufficient energy, the photon generates 
an electron-hole pair. If the reset voltage signal is brought LOW so that nMOSFET 310 is 
OFF but the shutter voltage signal is still HIGH, then electron-hole pairs generated by 
photons captured in diode 305 discharge diode's 305 parasitic capacitance, reducing 
diode's 305 reverse bias voltage and similarly reducing the voltage stored on capacitor 
10 320. The length of time for which the shutter voltage signal is HIGH and the reset voltage 
signal is LOW determines the integration time. In one embodiment, diode 305 is initially 
biased to approximately 2.3 volts. 

Fig. 4 illustrates various signals related to Fig. 3. In Fig. 4, the reset and shutter 
voltage signals are HIGH during time segment A-B, whereas the reset voltage signal is 
1 5 OFF during time segment B-C but the shutter voltage signal is HIGH. Time segment B-C 
represents the integration time. During the integration time, the voltage at the cathode of 
diode 305 and the voltage of capacitor 320 are decaying due to the generation of electron- 
hole pairs. After the shutter voltage signal goes LOW at time C, the cathode voltage of 
diode 305 continues to decay, but the voltage of capacitor 320 stays approximately 
20 constant. The voltage difference denoted by V^ge in Fig. 4, which is the difference 

between the initial and final voltages of capacitor 320, represents the time integration of 
the intensity of light captured by pixel diode 305 during a frame time. 

To determine V image , the voltage of capacitor 320 needs to be read when diode 305 
is fully reverse biased (when the reset voltage signal is HIGH) and when the shutter 
25 voltage signal returns LOW. The voltage of capacitor 320 is read at the source of 
nMOSFET 325, where nMOSFET 325, current mirror nMOSFET s 330, and current 
source 335 comprise a follower. The source voltage of nMOSFET 325 is read by 
switching a row address voltage signal HIGH so that nMOSFETs 340 are ON. If column 
line 345 is properly discharged before the row address voltage signal is HIGH, then the 
30 voltage on column line 345 is to within a threshold voltage drop of the voltage of 
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capacitor 320. Not shown in Fig. 3 are other column pixels coupled to column line 345 
and identical in structure to that shown in Fig. 3. 

The row address voltage signal is shown in Fig. 4. By setting the row address 
voltage signal HIGH during the time segment A-B, the source voltage of nMOSFET 325 

5 is read when capacitor 320 is folly charged, and by setting the row address voltage signal 
HIGH during the time segment C-F (but not at time F) the source voltage of nMOSFET 
325 is read when capacitor 320 is discharged after the integration time. The former and 
latter read source voltages provide calibration and sample voltage signals, respectively. If 
dark voltage was not present, then Vi mage would be an accurate representation of the time 

10 integration of the light intensity incident upon pixel diode 305. Subtracting the sample 
voltage signal from the calibration voltage signal compensates for pixel-to-pixel variation, 
such as differences in threshold voltages for the source follower transistors (nMOSFET 
325). 

Fig. 3 is only one representative example of a pixel sensor. The output signals from 
15 the pixel sensors are processed to obtain interpolated red, green, and blue color 

component signals. Let R{n,m), G(«,m), and B{n,rn) denote, respectively, interpolated 
red, green, and blue color component signals for row-column position (n,m) . 

A method for providing the interpolated color component signals for the tiling 
pattern of Fig. 2 can be concisely described by introducing the notation that an index / for 
20 which / = 1, 2, or 3, denotes, respectively, red, blue, or green. As an example of this 
notation, let X l (n,m) denote an interpolated color component signal, where 
X x {n,m) = R(n,m), X 2 (n,m) = G(n,m) , and X 3 (n,m) = B(n 7 m) . Amethodfor 
providing the interpolated color component signals for the tiling pattern of Fig. 2 is now 
described as follows: For a position (n 7 m) , pixel sensor output signal X(n,m) , and for 
25 each / = 1, 2, 3, if X{n,m) is an output signal of a color / pixel sensor, then 

X x (n,m) = X{n,m) , and if X(n,m) is an output signal of an IR pixel sensor or a color 
j * i pixel sensor, thenZ, (n,m) is an average of the output signals of nearest neighbor 
color / pixel sensors. 
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Fig. 5 provides a flow diagram of the above method. In step 502, the averaging is 
understood to be over nearest neighbor pixel sensor positions having color /. To illustrate 
examples, refer to Fig. 6, which is the tiling pattern of Fig. 2 extended to have five rows. 
The rows and columns in Fig. 6 are indexed as shown for convenience only. As an 
example of nearest neighbor averaging, consider position (2,2) in Fig. 6. This is an IR pass 
filter. A set of interpolated color signal components is: 

X(l,3) + X(3,l) 



R(2,2) 
G(2,2) = 
3(2,2) = 



2 

X(l,2) + X(2,l) + X(3,2) + X(2,3) 
4 

X(\,l) + X(3,3) 



2 

As another example, consider position (3,3) in Fig. 6. This is a blue pass filter. A set of 
interpolated color signal components is: 

X(l,3) + Jf(5,3) 



i?(3,3) = 

10 G(3,3) = 



2 

X(2,3) + X(3,2) + X(3,4) + X(4,3) 



4 

5(3,3) = X(3,3). 

For another example, consider position (3,2) in Fig. 6. This is a green pass filter. A set of 
interpolated color signal components is: 

i?(3,2) = X(3,l), 

G(3,2) = X(3,2), 

5(3,2) = X(3,3). 

The above sums and divisions in the above displayed equations are to be 
15 interpreted as operations performed in either the analog or digital domain. In general, the 
averaging of signals over nearest neighbors is approximate because of finite wordlengths, 
if performed in the digital domain, or the inherent approximate nature of performing 
operations in the analog domain. Furthermore, the interpolated signals need only be 
proportional (or approximately) proportional to an average. For example, the right-hand- 
20 sides of the above displayed equations may be multiplied by a proportionality factor. 

The averaging operations may be performed by non-programmed hardwired 
circuits (either analog or digital), by circuits programmed by firmware (software stored in 
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non-volatile memory), by a computer, or any combination of such methods. For example, 
a simplified computer system is illustrated in Fig. 7. In the embodiment of Fig. 7, 
processor 702 performs the nearest neighbor averaging indicated in step 502 of Fig. 5, 
where the software code for performing this averaging is stored in memory 704. In Fig. 7, 
5 camera provides image data to processor 702. Video interface circuit 708, which for some 
embodiments may not be needed, provides for communication between processor 702 and 
camera 706 via bus 710. As one alternative to the embodiment of Fig. 7, pixel averaging 
may be performed by circuits within camera 706. 

Returning to Fig. 5, in step 504, the indices n and m are changed and new 

10 interpolated color component signals are interpolated for a new position (n,m) . The 
processing indicated by Fig. 5 may be performed on the pixel sensor positions in a 
sequential manner, or more than one position may be processed in parallel. 

The output signals from the IR pixel sensors may be used to subtract the IR 
components from the X i (n 7 m) . For example, an interpolated IR signal component for 

15 position (n 7 m) may be obtained by taking the average of the output signals from the 

nearest IR pixel sensors, and subtracting this interpolated IR signal from the interpolated 
color component signals. For some embodiments, this processing may be performed by 
processor 702. 

The IR pass filters may be manufactured using commercial Color Filter Array 
20 (CFA) materials, taking advantage of the fact that these materials are transparent to IR 
radiation. By a simple overlay of two CFA colors (e.g., R and B pass filters) that have no 
overlapping transmittance in the visible portion of the spectrum, it is possible to create a 
composite filter element that blocks the visible light and transmits only IR. If two filters 
are used to form the composite filter, then each of the two filters has a visible radiation 
25 pass spectrum that is disjoint from the other, so that there is substantially no transmittance 
of visible light through the resulting composite filter formed from the combination of the 
two filters. If more than two filters are used, then each filter has a visible radiation pass 
spectrum such that the resulting composite filter is substantially opaque to visible light. 
This composite filter element is thus an IR pass filter, because each of the component 
30 filters used to form the composite filter is substantially transparent to IR. 
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Another embodiment of the present invention for the CMY (Cyan- Magenta- 
Yellow) color space is provided in Fig. 8. Fig. 8 denotess a unit array of filters for a tiling 
pattern, where filters designated by C, Y, and M denote pass filters for the colors cyan, 
yellow, and magenta, respectively. Note that the unit array of Fig. 8 corresponds ot the 
5 unit array of Fig. 2 by making the correspondances: CoBJoG, and M <e> R. 

Fig. 5 provides a flow diagram for pixel interpolation with the CMY tiling pattern 
of Fig. 8, where now M(n 7 m) , Y{n,m), and Cin^m) denote, respectively, interpolated 
magenta, yellow, and cyan color component signals for row-column position (n,m) , and 
now X t {n,m) denotes an interpolated color component signal where 
1 0 X x (n y m) - M(n, m) , X 2 (n, m) = Y{n, m) , and X 3 (n, m) = C{n, m) . 

It is to be understood that in some embodiments, not all interpolated color 
component signals need be obtained by averaging signals from nearest neighbors with like 
colors. For example, signals from pixel sensors at the edge of the CFA may be processed 
differently. Or, only a proper subset of the signal outputs from the pixel sensors may be 
15 interpolated. Many modifications may be made to the above disclosed embodiments, and 
the invention is limited only by the scope of the claims below. 
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What is claimed is: 



1 . A color filter array comprising a unit array, the unit array having green, red, blue, 
and infrared pass filters in the relative ratios of 4: 1 : 1 :2, respectively. 

5 

2. A color filter array comprising: 

a first row of pass filters, comprising, in order, blue, green, red, and green pass 

filters; 

a second row of pass filters, comprising, in order, green, infrared, green, and 
10 infrared pass filters, wherein the second row of pass filters is adjacent to the first row of 
pass filters so that the blue pass filter of the first row is adjacent to the first green pass 
filter of the second row; 

a third row of pass filters, comprising, in order, red, green, blue, and green pass 
filters, wherein the first green pass filter of the second row is adjacent to the red pass filter 
15 of the third row; and 

a fourth row of pass filters, comprising, in order, green, infrared, green, and 
infrared pass filters, wherein the red pass filter of the third row is adjacent to the first 
green pass filter of the fourth row. 

20 3. A color filter array comprising an array of pass filters / wherein for some n and 

m: 

A+um+i md L + 3^3 are Wue pass filters; 

•/w+l.m+2 5 fn+\,m+4 ? fn+2,m+\,fn+2,m+3 ? fn+3,m+2 ' fn+3,m+4 ? fn+4,m+\ ? fn+4,m+3 green 

pass filters; 

25 fn +hm+ 3 and f n+3 m+l are red pass filters; and 

fn+2,m*2>f*+2.**>fn+**»2> md L,m4 infrared pass filters. 

4. An imaging system comprising: 

a color filter array comprising an array of pass filters / wherein for some n and m : 
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A + i, m+ i and/ rt+3 m+3 are blue pass filters; 

fn+\,m+2 ? fn+\,m+4 > fn+2,m+\,fn+2,m+3 -> fn+3,m+2 *> fn+3,m+4 ' fn+4,m+l ' ai *^ f n+ ^ >m ^ green 

pass filters; 

L + im + 3 and /„ +3 m+1 are red pass filters; 

5 , A +2?m+ 4 , 7„ +4sm+ 2 , and f nW are infrared pass filters; and 

an array of pixel sensors responsive to electromagnetic radiation propagating 
through the color filter array, wherein for some range of position indices u and v, a pixel 
sensor at position (w,v) provides an output signal X(u,v) indicative of electromagnetic 
radiation propagating through the color filter array and impinging upon the pixel sensor at 

10 position (w,v) . 



5. The imaging system as set forth in claim 4, further comprising 

at least one processor to provide interpolated color component signals X t ( u > v )> i 
= 1, 2, 3, where / = 1 denotes red, / = 2 denotes green, and i = 3 denotes blue, wherein for 
15 each i = 1, 2 5 3; if X(u, v) is an output signal of a color / pixel sensor, then 

X t (h, v) = X(u, v) , and if X(u, v) is an output signal of an IR pixel sensor or a color 
j it i pixel sensor, then X t (u,v) is an average of the output signals of nearest neighbor 
color / pixel sensors. 

20 6. The imaging system as set forth in claim 4, further comprising 

a memory storage device, wherein stored in the memory storage device are 
instructions to process interpolated color component signals X t (u,v), i = 1, 2, 3, where / 
= 1 denotes red, / = 2 denotes green, and i = 3 denotes blue, wherein for each / = 1, 2, 3; if 
X(u,v) is an output signal of a color i pixel sensor, then X i (u,v) = X(u,v) , and if 

25 X(u 9 v) is an output signal of an IR pixel sensor or a color j ^ / pixel sensor, 

thenX, (z/, v) is an average of the output signals of nearest neighbor color / pixel sensors. 
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7. A method to interpolate color component signals, comprising: 
providing a color filter array comprising 

a first row of pass filters, comprising, in order, blue, green, red, and green 

pass filters; 

a second row of pass filters, comprising, in order, green, infrared, green, 
and infrared pass filters, wherein the second row of pass filters is adjacent to the first row 
of pass filters so that the blue pass filter of the first row is adjacent to the first green pass 

filter of the second row; 

a third row of pass filters, comprising, in order, red, green, blue, and green 
pass filters, wherein the first green pass filter of the second row is adjacent to the red pass 
filter of the third row; and 

a fourth row of pass filters, comprising, in order, green, infrared, green, 
and infrared pass filters, wherein the red pass filter of the third row is adjacent to the first 
green pass filter of the fourth row; 

providing an array of pixel sensors responsive to electromagnetic radiation 
propagating through the color filter array, wherein for some range of position indices u 
and v, a pixel sensor at position (w,v) provides an output signal X(u,v) indicative of 
electromagnetic radiation propagating through the color filter array and impinging upon 
the pixel sensor at position (u, v) ; and 

interpolating to provide interpolated color component signals X, (u,v), i = 1, 2, 3, 
where i = 1 denotes red, i = 2 denotes green, and / = 3 denotes blue, wherein for each / = 
1, 2, 3; if X(u,v) is an output signal of a color i pixel sensor, then X, (u,v) = X(u, v) , 
and if X(u, v) is an output signal of an IR pixel sensor or a color j * i pixel sensor, 
then J£,(tf,v) is an average of the output signals of nearest neighbor color / pixel sensors. 

8. A method to interpolate color component signals, comprising: 
providing a color filter array comprising 

a color filter array comprising an array of pass filters / wherein for some n 

and m: 
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and f„ +3 m+s are blue pass filters; 

/n+l,m+2>/n+1,m+4»/«+2,m+l,/rt+2 > ra+3'/n+3,m+2 > /n+3,m+4 > / i+4,m+l > and / , +4>m+3 are 

green pass filters; 

A +1 , m+ 3 and /„ +3>m+1 are red pass filters; and 

5 /„ +2 , m+ 2 , /» + 2, m+ 4 , / n+ 4, m+ 2 > and /„ +4 , m+4 are infrared pass filters; 

providing an array of pixel sensors responsive to electromagnetic radiation 
propagating through the color filter array, wherein for some range of position indices u 
and v, a pixel sensor at position (w,v) provides an output signal X(u,v) indicative of 
electromagnetic radiation propagating through the color filter array and impinging upon 

10 the pixel sensor at position (u,v) ; and 

interpolating to provide interpolated color component signals i",(w,v), /' = 1, 2, 3, 
w here / = 1 denotes red, / = 2 denotes green, and / = 3 denotes blue, wherein for each i = 
1, 2, 3; if X(u,v) is an output signal of a color z pixel sensor, then X t (u,v) = X(u,v) , 
and if X(u, v) is an output signal of an IR pixel sensor or a color j * i pixel sensor, 

1 5 then X t (u, v) is an average of the output signals of nearest neighbor color /' pixel sensors. 

9. A color filter array comprising a unit array, the unit array having yellow, magenta, 
cyan, and infrared pass filters in the relative ratios of 4: 1 : 1 :2, respectively. 

20 10. A color filter array comprising: 

a first row of pass filters, comprising, in order, cyan, yellow, magenta, and yellow 
pass filters; 

a second row of pass filters, comprising, in order, yellow, infrared, yellow, and 
infrared pass filters, wherein the second row of pass filters is adjacent to the first row of 
25 pass filters so that the cyan pass filter of the first row is adjacent to the first yellow pass 
filter of the second row; 
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a third row of pass filters, comprising, in order, magenta, yellow, cyan, and yellow 
pass filters, wherein the first yellow pass filter of the second row is adjacent to the 
magenta pass filter of the third row; and 

a fourth row of pass filters, comprising, in order, yellow, infrared, yellow, and 
5 infrared pass filters, wherein the magenta pass filter of the third row is adjacent to the first 
yellow pass filter of the fourth row. 

11. A color filter array comprising an array of pass filters f hJ wherein for some n and 
m: 

10 f nW and / n+3 m+3 are cyan pass filters; 

/n+1,m+2 >/n+l,m+4 ,fn+2,m+\,fn+2,m+3>fn+3,m+2>fn+3,m+4>f *+4,m+l > aTl ^ f n+4,m+3 aTQ y e U° W 

pass filters; 

f nM and /„ +3>m+1 are magenta pass filters; and 
f n+ 2,^Jn + 2^J n ,4, m+ 2 , and /„ +4 , M+4 are infrared pass filters. 

15 

12. An imaging system comprising: 

a color filter array comprising an array of pass filters / wherein for some n and m: 

/„ + i, m+ i and f n+Xm+i are cyan pass filters; 

fn+l,m+2 >fn+l,m+4>fn+2,m+\,fn+2,m+3>fn+3,m+2 >/n+3,m+4>/ «+4,m+\ > f »+4,m+3 yellOW 

20 pass filters; 

A + i, m+ 3 and/„ +3 m+1 are magenta pass filters; 

/„ + 2,m + 2 . /n + 2, m+ 4 , fn + 4,m + 2 , fn + 4, m+ 4 *™ P* SS filte^ and 

an array of pixel sensors responsive to electromagnetic radiation propagating 
through the color filter array, wherein for some range of position indices u and v, a pixel 
25 sensor at position (u,v) provides an output signal X(u,v) indicative of electromagnetic 
radiation propagating through the color filter array and impinging upon the pixel sensor at 
position (w,v). 
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13. The imaging system as set forth in claim 12, farther comprising 

at least one processor to provide interpolated color component signals X t (u,v) , / 
= 1, 2, 3, where / = 1 denotes magenta, / = 2 denotes yellow, and / = 3 denotes cyan, 
wherein for each i = 1, 2, 3; if X(u, v) is an output signal of a color i pixel sensor, then 

5 X i (w, v) = X(u 7 v) , and if X(u 9 v) is an output signal of an IR pixel sensor or a color 
j * i pixel sensor, then X l (u, v) is an average of the output signals of nearest neighbor 
color i pixel sensors. 

14. The imaging system as set forth in claim 12, further comprising 

10 a memory storage device, wherein stored in the memory storage device are 

instructions to process interpolated color component signals X r (w,v), / = 1, 2, 3, where / 
= 1 denotes magenta, / = 2 denotes yellow, and i = 3 denotes cyan, wherein for each / = 1, 
2, 3; if X(u, v) is an output signal of a color i pixel sensor, then X 1 0, v) = X(u, v) , and if 
X(u, v) is an output signal of an IR pixel sensor or a color j * i pixel sensor, 

15 then Jt j (u,v) is an average of the output signals of nearest neighbor color / pixel sensors. 

15. A method to interpolate color component signals, comprising: 
providing a color filter array comprising 

a first row of pass filters, comprising, in order, cyan, yellow, magenta, and 
20 yellow pass filters; 

a second row of pass filters, comprising, in order, yellow, infrared, yellow, 
and infrared pass filters, wherein the second row of pass filters is adjacent to the first row 
of pass filters so that the cyan pass filter of the first row is adjacent to the first yellow pass 
filter of the second row; 

25 a third row of pass filters, comprising, in order, magenta, yellow, cyan, and 

yellow pass filters, wherein the first yellow pass filter of the second row is adjacent to the 
magenta pass filter of the third row; and 
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a fourth row of pass filters, comprising, in order, yellow, infrared, yellow, 
and infrared pass filters, wherein the magenta pass filter of the third row is adjacent to the 
first yellow pass filter of the fourth row; 

providing an array of pixel sensors responsive to electromagnetic radiation 
5 propagating through the color filter array, wherein for some range of position indices u 
and v, a pixel sensor at position 0,v) provides an output signal X(u 9 v) indicative of 
electromagnetic radiation propagating through the color filter array and impinging upon 
the pixel sensor at position (w, v) ; and 

interpolating to provide interpolated color component signals X t (u,v), i = 1, 2, 3, 
10 where / = 1 denotes magenta, / = 2 denotes yellow, and i = 3 denotes cyan, wherein for 
each / = 1, 2, 3; if X(u,v) is an output signal of a color / pixel sensor, then 
X t (if, v) = X(u, v) , and if X(u,v) is an output signal of an IR pixel sensor or a color 
j * i pixel sensor, then X t (w, v) is an average of the output signals of nearest neighbor 
color / pixel sensors. 

15 

16. A method to interpolate color component signals, comprising: 
providing a color filter array comprising 

a color filter array comprising an array of pass filters / wherein for some n 

and m\ 

20 f n+hm+ , and f n+Xm+3 are cyan pass filters; 

fn+\,m+2 ?fn+\,m+4 >fn+2,m+\,fn+2,m+3>fn+3,m+2 ?fn+2,m+4>f n+4,m+\ ? f n+4 7 m+3 ai " C 

yellow pass filters; 

f n+hm+3 and f n+Xm+l are magenta pass filters; and 

/ H+ 2.^2, L + 2, m+ 4J n +4,m + 2, ^ L + 4,m + 4 infrared pass filters; 
25 providing an array of pixel sensors responsive to electromagnetic radiation 

propagating through the color filter array, wherein for some range of position indices u 
and v, a pixel sensor at position (u,v) provides an output signal X(u,v) indicative of 



i 
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electromagnetic radiation propagating through the color filter array and impinging upon 
the pixel sensor at position (w,v) ; and 

interpolating to provide interpolated color component signals X t (u,v), = 1, 2, 3, 
where /' = 1 denotes magenta, i = 2 denotes yellow, and i = 3 denotes cyan, wherein for 
each i = 1, 2, 3; if X(u,v) is an output signal of a color i pixel sensor, then 
X, (w,v) = X(u,v) , and if X(u, v) is an output signal of an IR pixel sensor or a color 
j ± i pixel sensor, thenX, (w, v) is an average of the output signals of nearest neighbor 
color i pixel sensors. 
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Abstract 

An imager with a four color mosaic pattern of red, green, blue, and infrared pass 
filters, where color component signals for a pixel are interpolated by averaging over 
nearest neighbor pixels. 
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DECLARATION AND POWER OF ATTORNEY FOR PATENT APPLICATION 
(FOR INTEL CORPORATION PATENT APPLICATIONS) 

As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below, next to my name. 

I believe I am the original, first, and sole inventor (if only one name is listed below) or an 
original, first, and joint inventor (if plural names are listed below) of the subject matter 
which is claimed and for which a patent is sought on the invention entitled: 

COLOR INTERPOLATION FOR A FOUR COLOR MOSAIC PATTERN 

the specification of which 

XXX is attached hereto. 

was filed on as 

United States Application Number 

or PCT International Application Number 

and was amended on ■ 

(if applicable) 

I hereby state that I have reviewed and understand the contents of the above-identified 
specification, including the claim(s), as amended by any amendment referred to above. I 
do not know and do not believe that the claimed invention was ever known or used in the 
United States of America before my invention thereof, or patented or described in any 
printed publication in any country before my invention thereof or more than one year 
prior to this application, that the same was not in public use or on sale in the United States 
of America more than one year prior to this application, and that the invention has not 
been patented or made the subject of an inventor's certificate issued before the date of this 
application in any country foreign to the United States of America on an application filed 
by me or my legal representatives or assigns more than twelve months (for a utility 
patent application) or six months (for a design patent application) prior to this 
application. 

I acknowledge the duty to disclose all information known to me to be material to 
patentability as defined in Title 37, Code of Federal Regulations, Section 1 .56. 

I hereby claim foreign priority benefits under Title 35, United States Code, Section 
119(a)-(d), of any foreign application(s) for patent or inventor's certificate listed 
below and have also identified below any foreign application for patent or inventor's 
certificate having a filing date before that of the application on which priority is claimed: 
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Prior Foreign Application(s) 



Priority 
Claimed 



(Number) (Country) (Day/Month/Year Filed) Yes No 



(Number) (Country) (Day/Month/Year Filed) Yes No 



(Number) (Country) (Day/Month/Year Filed) Yes No 

I hereby claim the benefit under title 35, United States Code, Section 119(e) of any United 
States provisional application(s) listed below 



(Application Number) Filing Date 



(Application Number) Filing Date 

I hereby claim the benefit under Title 35, United States Code, Section 120 of any United 
States application(s) listed below and, insofar as the subject matter of each of the claims 
of this application is not disclosed in the prior United States application in the manner 
provided by the first paragraph of Title 35, United States Code, Section 112, I 
acknowledge the duty to disclose all information known to me to be material to 
patentability as defined in Title 37, Code of Federal Regulations, Section 1 .56 which 
became available between the filing date of the prior application and the national or PCT 
international filing date of this application: 



(Application Number) Filing Date (Status -- patented, 

pending, abandoned) 



(Application Number) Filing Date (Status -- patented, 

pending, abandoned) 
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I hereby appoint Farzad E. Amini, Reg. No. P42.261; Aloysius T. C. AuYeung, Reg. No. 35,432; 
Amy M. Armstrong, Reg. No. P42,265; William Thomas Babbitt, Reg. No. 39,591; Carol F. Barry, 
Reg. No. P41.600; Jordan Michael Becker, Reg. No. 39,602; Bradley J. Bereznak, Reg. No. 33,474; 
Michael A. Bernadicou, Reg. No. 35,934; Roger W. Blakely, Jr., Reg. No. 25,831; Gregory D. 
Caldwell, Reg. No. 39,926; Kent M. Chen, Reg. No. 39,630; Lawrence M. Cho, Reg. No. 39,942; 
Yong S. Choi, Reg. No. P43,324; Thomas M. Coester, Reg. No. 39,637; Roland B. Cortes, Reg. No. 
39,152; Barbara Bokanov Courtney, Reg. No. P42.442; Michael Anthony DeSanctis, Reg. No. 
39,957; Daniel M. De Vos, Reg. No. 37,813; Robert Andrew Diehl, Reg. No. 40,992; Tarek N. 
Fahmi, Reg. No. 41,402; James Y. Go, Reg. No. 40,621; Richard Leon Gregory, Jr., P42,607; 
Dinu Gruia, Reg. No. P42.996; David R. Halvorson, Reg. No. 33,395; Thomas A. Hassing, Reg. 
No. 36,159; Phuong-Quan Hoang, P41,839; Willmore F. Holbrow III, Reg. No. P41.845; 
George W Hoover II, Reg. No. 32,992; Eric S. Hyman, Reg. No. 30,139; Dag H. Johansen, Reg. 
No. 36,172; William W. Kidd, Reg. No. 31,772; Michael J. Mallie, Reg. No. 36,591; Andre L. 
Marais, under 37 C.F.R. § 10.9(b); Paul A. Mendonsa, Reg. No. P42.879; Darren J. Milliken, 
P42.004; Thinh V. Nguyen, P42.034; Kimberley G. Nobles, Reg. No. 38,255; Michael A. 
Proksch, Reg. No. P43.021; Babak Redjaian, P42.096; James H. Salter, Reg. No. 35,668; 
William W. Schaal, Reg. No. 39,018; James C. Scheller, Reg. No. 31,195; Anand Sethuraman, 
Reg. No. P43.351; Charles E. Shemwell, Reg. No. 40,171; Maria McCormack Sobrino, Reg. No. 
31,639; Stanley W. Sokoloff, Reg. No. 25,128; Allan T. Sponseller, Reg. No. 38,318; Judith A. 
Szepesi, Reg. No. 39,393; Vincent P. Tassinari, Reg. No. P42.179; Edwin H. Taylor, Reg. No. 
25,129; George G. C. Tseng, Reg. No. 41,355; Lester J. Vincent, Reg. No. 31,460; John 
Patrick Ward, Reg. No. 40,216; Stephen Warhola, Reg. No. P43.237; Ben J. Yorks, Reg. No. 
33,609; and Norman Zafman, Reg. No. 26,250; my attorneys, of BLAKELY, SOKOLOFF, TAYLOR 
& ZAFMAN LLP, with offices located at 12400 Wilshire Boulevard, 7th Floor, Los Angeles, 
California 90025, telephone (310) 207-3800, and Alan K. Aldous, Reg. No. 31,905; Robert 
D. Anderson, Reg. No. 33,826; Joseph R. Bond, Reg. No. 36,458; Richard C. Calderwood, Reg. 
No. 35,468; Cynthia Thomas Faatz, Reg No. 39,973; Sean Fitzgerald, Reg. No. 32,027; Seth Z. 
Kalson, Reg. No. 40,670; David J. Kaplan, Reg. No. 41,105; Leo V. Novakoski, Reg. No. 37,198; 
Naomi Obinata, Reg. No. 39,320; Thomas C. Reynolds, Reg. No. 32,488; Steven P. Skabrat, Reg. 
No. 36,279; Howard A. Skaist, Reg. No. 36,008; Steven C. Stewart, Reg. No. 33,555; Raymond 
J. Werner, Reg. No. 34,752; and Charles K. Young, Reg. No. 39,435; my patent attorneys, and 
Jeffrey S. Draeger, Reg. No. 41,000; Thomas Raleigh Lane, Reg. No. P42,781; Calvin E. Wells, 
Reg. No. P43.256; and Alexander Ulysses Witkowski, Reg. No. P43,280; my patent agents, of 
INTEL CORPORATION; and James R. Thein, Reg. No. 31,710, my patent attorney; with full power 
of substitution and revocation, to prosecute this application and to transact all business in the 
Patent and Trademark Office connected herewith. 



Send correspondence to James H. Salter , BLAKELY, SOKOLOFF, TAYLOR & 

(Name of Attorney or Agent) 
ZAFMAN LLP, 12400 Wilshire Boulevard, 7th Floor, Los Angeles, California 90025 and 
direct telephone calls to James H. Salter (408) 720-8598. 

(Name of Attorney or Agent) 



INTEL CORPORATION 



-3- 



42390. P6376 



I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false statements and the like so made 
are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the 
United States Code and that such willful false statements may jeopardize the validity of the 
application or any patent issued thereon. 

Full Name of Sole/First Inventor Tinku Acharva _ 

Inventor's Signature ___ Date 

Citizenship 

(Country) 

Post Office Address (Business or Residence or P.O. Box) _ 



Indicate below with an "X" whether the Post Office Address set forth above is either: 
Residence Address or 

Business Address or other address where mail is customarily received (e.g., P.O. Box). 

If the Post Office Address set forth above is not a residence address, then provide the City 

and State of Residence 

(City and State of Residence) 

Full Name of Sole/Second Inventor Edward J. Bawolek 

Inventor's Signature _ Date . 

Citizenship 

(Country) 

Post Office Address (Business or Residence or P.O. Box) 



Indicate below with an "X" whether the Post Office Address set forth above is either: 
Residence Address or 

Business Address or other address where mail is customarily received (e.g., P.O. Box). 

If the Post Office Address set forth above is not a residence address, then provide the City 

and State of Residence _ . 

(City and State of Residence) 

Full Name of Sole/Third Inventor Pina-Sina Tsai . 

Inventor's Signature Date _ 

Citizenship 

(Country) 

Post Office Address (Business or Residence or P.O. Box) . 



Indicate below with an "X" whether the Post Office Address set forth above is either: 
Residence Address or 

Business Address or other address where mail is customarily received (e.g., P.O. Box). 

If the Post Office Address set forth above is not a residence address, then provide the City 

and State of Residence 

(City and State of Residence) 
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Full Name of Sole/Fourth Inventor John J. Bean 

Inventor's Signature 

Citizenship 

(Country) 

Post Office Address (Business or Residence or P.O. Box) 



Indicate below with an "X" whether the Post Office Address set forth above is either: 
Residence Address or 

Business Address or other address where mail is customarily received (e.g., P.O. Box). 

If the Post Office Address set forth above is not a residence address, then provide the City 

and State of Residence _ 

(City and State of Residence) 



Full Name of Sole/Fifth Inventor 

Inventor's Signature Date 

Citizenship __ 

(Country) 

Post Office Address (Business or Residence or P.O. Box) 



Indicate below with an "X" whether the Post Office Address set forth above is either: 
Residence Address or 

Business Address or other address where mail is customarily received (e.g., P.O. Box). 

If the Post Office Address set forth above is not a residence address, then provide the City 

and State of Residence . 

(City and State of Residence) 



Full Name of Sole/Sixth Inventor 

Inventor's Signature Date 

Citizenship 

(Country) 

Post Office Address (Business or Residence or P.O. Box) 



Indicate below with an "X" whether the Post Office Address set forth above is either: 
Residence Address or 

Business Address or other address where mail is customarily received (e.g., P.O. Box). 

If the Post Office Address set forth above is not a residence address, then provide the City 

and State of Residence 

(City and State of Residence) 
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Title 37, Code of Federal Regulations, Section 1.56 
Duty to Disclose Information Materi al to Patentability 



(a) A patent by its very nature is affected with a public interest. The public interest is best served, 
and the most effective patent examination occurs when, at the time an application is being examined, the 
Office is aware of and evaluates the teachings of all information material to patentability. Each individual 
associated with the filing and prosecution of a patent application has a duty of candor and good faith in dealing 
with the Office, which includes a duty to disclose to the Office ail information known to that individual to be 
material to patentability as defined in this section. The duty to disclosure information exists with respect to 
each pending claim until the claim is cancelled or withdrawn from consideration, or the application becomes 
abandoned. Information material to the patentability of a claim that is cancelled or withdrawn from 
consideration need not be submitted if the information is not material to the patentability of any claim remaining 
under consideration in the application. There is no duty to submit information which is not material to the 
patentability of any existing claim. The duty to disclosure all information known to be material to patentability 
is deemed to be satisfied if all information known to be material to patentability of any claim issued in a patent 
was cited by the Office or submitted to the Office in the manner prescribed by §§1 -97(b)-(d) and 1 .98. 
However, no patent will be granted on an application in connection with which fraud on the Office was practiced 
or attempted or the duty of disclosure was violated through bad faith or intentional misconduct. The Office 
encourages applicants to carefully examine: 

(1) Prior art cited in search reports of a foreign patent office in a counterpart application, and 

(2) The closest information over which individuals associated with the filing or prosecution of a 
patent application believe any pending claim patentably defines, to make sure that any material information 
contained therein is disclosed to the Office. 

(b) Under this section, information is material to patentability when it is not cumulative to 
information already of record or being made or record in the application, and 

(1) It establishes, by itself or in combination with other information, a prima facie case of 
unpatentability of a claim; or 

(2) It refutes, or is inconsistent with, a position the applicant takes in: 

(i) Opposing an argument of unpatentability relied on by the Office, or 

(ii) Asserting an argument of patentability. 

A prima facie case of unpatentability is established when the information compels a conclusion that a claim is 
unpatentable under the preponderance of evidence, burden-of-proof standard, giving each term in the claim its 
broadest reasonable construction consistent with the specification, and before any consideration is given to 
evidence which may be submitted in an attempt to establish a contrary conclusion of patentability. 

(c) Individuals associated with the filing or prosecution of a patent application within the 
meaning of this section are: 

(1 ) Each inventor named in the application; 

(2) Each attorney or agent who prepares or prosecutes the application; and 

(3) Every other person who is substantively involved in the preparation or prosecution of the 
application and who is associated with the inventor, with the assignee or with anyone to whom there is an 
obligation to assign the application. 

(d) Individuals other than the attorney, agent or inventor may comply with this section by 
disclosing information to the attorney, agent, or inventor. 
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